Single and multi-sensor cameras are predominant in image or video capture systems. The cameras typically employ CCD or CMOS sensors that utilize a Bayer pattern color filter array (where each pixel only contains one color value). When using a Bayer pattern color filter array, the captured image is a mosaic of red, green, and blue (RGB) colors where each pixel is missing two of the three color values. Accordingly, these missing color values need to be interpolated to obtain an approximation of the full RGB values for each pixel to provide a richer output image.
Vendors can provide different methods to decode the source image to the full RGB patterns in the output image, including bilinear reconstruction, nearest neighbor, cubic, cubic spline, etc., all which have different computational speed and accuracy tradeoffs. As handheld devices become smaller, the amount of available memory and computing power can become limiting factors while consumer demand remains high for better image and video quality.
However, many of the reconstruction processes for images are based on the full RGB pattern; therefore, memory size and communication bandwidth remain big issues and are not easy to implement in hardware.